Method and System for Automated Defect Detection

ABSTRACT

A computer program product and method for performing automated defect detection of blades within an engine is disclosed. The method may include providing a storage medium for storing data and programs used in processing video images, providing a processing unit for processing images, receiving from a borescope an initial set of images of a plurality of members inside of a device, and using the processing unit to apply Robust Principal Component Analysis to decompose the initial set of images into a first series of low rank component images and a second series of sparse component images, wherein there are at least two images in the initial series.

TECHNICAL FIELD OF THE DISCLOSURE

The present disclosure relates to automated inspection techniques and,more particularly, relates to automated visual inspection techniques ofimages or videos captured by image capture devices such as borescopes.

BACKGROUND OF THE DISCLOSURE

Video inspection systems, such as borescopes, have been widely used forcapturing images or videos of difficult-to-reach locations by “snaking”image sensor(s) to these locations. Applications utilizing borescopeinspections include aircraft engine blade inspection, power turbineblade inspection, internal inspection of mechanical devices, and thelike.

A variety of techniques for inspecting the images or videos provided byborescopes for determining defects therein have been proposed in thepast. Most such techniques capture and display images or videos to humaninspectors for defect detection and interpretation. Human inspectorsthen decide whether any defect within those images or videos exists.When human inspectors look at many similar images of very similar bladesof an engine stage, sometimes they miss defects because of therepetitive nature of the process or because of physical fatigueexperienced by the inspector. Missing a critical defect may lead tocustomer dissatisfaction, transportation of an expensive engine back toservice centers, lost revenue, or even engine failure.

Some other techniques utilize automated inspection techniques with manymanually-set detection thresholds that are error-prone in an automatedor semi-automated inspection system. In some of these other techniques,common defects are categorized into classes such as leading edgedefects, erosion, nicks, cracks, or cuts and any incoming images orvideos from the borescopes are examined to find those specific classesof defects. These techniques are thus focused on low-level featureextraction and identify damage by matching features and comparing tothresholds. Although somewhat effective, categorizing all kinds of bladedamage defects within classes is difficult and images having defectsother than those pre-defined classes are not detected.

Accordingly, it would be beneficial if an improved technique forperforming borescope inspections were developed. It would additionallybe beneficial if such a technique were automated, thus minimizing humanintervention and the multiplicity of manually tuned thresholds.

SUMMARY OF THE DISCLOSURE

In accordance with one aspect of the present disclosure, a method ofperforming automated defect detection is disclosed. The method mayinclude providing a storage medium for storing data and programs used inprocessing images, providing a processing unit for processing theimages, receiving from an image capture device an initial set of imagesof a plurality of members inside of a device, and using the processingunit to apply Robust Principal Component Analysis to decompose theinitial set of images into a first series of low rank component imagesand a second series of sparse component images, wherein there are atleast two images in the initial set.

In accordance with another aspect of the present disclosure, a methodfor performing automated defect detection on blades in an aircraftengine is disclosed. The method may include providing a storage mediumfor storing data and programs used in processing video images, providinga processing unit for processing video images, receiving from aborescope video images of a plurality of the blades of the engine, usingthe processing unit to decompose each of the video images using RobustPrincipal Component Analysis (RPCA) into a low rank matrix and a sparsematrix, and utilizing video image data in the sparse matrix to determinewhether there are possible defects within the plurality of blades.

In accordance with yet another aspect of the present disclosure, acomputer program product is disclosed. The computer program product maycomprise a computer usable medium having a computer readable programcode embodied therein. The computer readable program code may be adaptedto be executed to implement a method for performing automated defectdetection on blades in an aircraft engine. Such method may comprisereceiving from a borescope video images of a plurality of the blades ofthe engine, and decomposing each of the video images using RobustPrincipal Component Analysis into a low rank matrix and a sparse matrix,wherein data in the sparse matrix may be indicative of defects in theblades.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic illustration of an embodiment of an automatedborescope inspection system, in accordance with the present disclosure;

FIG. 2 is a flowchart illustrating exemplary steps of automated defectdetection in accordance with the present disclosure;

FIG. 3 illustrates an exemplary video image decomposed into low rankcomponent images and sparse component images;

FIGS. 4A-4D show an exemplary technique of further processing of thesparse image component data to confirm the presence of a defect; and

FIGS. 5A-5C show another exemplary technique of further processing ofthe sparse image component data to confirm the presence of a defect.

While the present disclosure is susceptible to various modifications andalternative constructions, certain illustrative embodiments thereof,will be shown and described below in detail. It should be understood,however, that there is no intention to be limited to the specificembodiments disclosed, but on the contrary, the intention is to coverall modifications, alternative constructions, and equivalents fallingwithin the spirit and scope of the present disclosure.

DETAILED DESCRIPTION OF THE DISCLOSURE

Referring to FIG. 1, a schematic illustration of an automated defectdetection system 2 is shown, in accordance with at least someembodiments of the present disclosure. In at least some embodiments, theautomated defect detection system 2 may be an automated borescopeinspection (ABI) system. As shown, the ABI system 2 may include anengine 4 having a plurality of stages 6, each of the stages having aplurality of blades 8, some or all of which may require visualinspection periodically, at predetermined intervals, or based on othercriteria by an image capture device 10. The engine 4 may berepresentative of a wide variety of engines such as jet aircraftengines, aeroderivative industrial gas turbines, steam turbines, dieselengines, automotive and truck engines, and the like. Notwithstanding thefact that the present disclosure has been described in relation tovisual inspection of the blades 8 of an engine 4, in other embodiments,the ABI system 2 may be employed to inspect other parts of the engine 4,as well as to perform inspection on the parts or members of other typesof equipment and devices. Such parts/members are not limited to blades.For example, the ABI system 2 may be used for processing medicalendoscope images, inspecting critical interior surfaces in machined orcast parts, forensic inspection, inspection of civil structures such asbuildings bridges, piping, etc.

The image capture device 10 may be an optical device having an opticallens or other imaging device or image sensor at one end and capable ofcapturing and transmitting images or videos through a communicationchannel 12 to a processing unit 14. In the preferred embodiment theimage capture device 10 may be representative of any of a variety offlexible borescopes or fiberscopes, rigid borescopes, video borescopesor other devices such as endoscopes which are capable of capturing andtransmitting images or videos of difficult-to-reach areas through thecommunication channel 12. The communication channel 12 in turn may be anoptical channel or alternatively, may be any other wired, wireless orradio channel or any other type of channel capable of transmittingimages and videos between two points including links involving the WorldWide Web (www) or the internet.

With respect to the processing unit 14, it may be located on-site nearor on the engine 4, or alternatively, it may be located at a remote siteaway from the engine 4. A storage medium 20 may be in communication withthe processing unit 14. The storage medium 20 may store data andprograms used in processing images or videos of the blades 8. Theprocessing unit 14 may receive and process images of the blades 8 thatare captured and transmitted by the image capture device 10 via thecommunication channel 12. Upon receiving the images, the processing unit14 may process the images to determine whether there are any defectswithin any of the blades 8.

Results (e.g., the defects) 18 may then be reported throughcommunication channel 16. The results 18 may include informationregarding whether any defects in any of the blades 8 were found.Information about the type of defect, the location of the defect, sizeof the defect, etc. may also be reported as part of the results 18.

Similar to the communication channel 12, the communication channel 16may be any of variety of communication links including, wired channels,optical or wireless channels, radio channels or possibly links involvingthe World Wide Web (www) or the internet. It will also be understoodthat although the results 18 have been shown as being a separate entityfrom the processing unit 14, this need not always be the case. Rather,in at least some embodiments, the results 18 may be stored within andreported through the processing unit 14 as well. Furthermore, reportingof the results 18 may involve storing the results in the storage medium20 for future reference, as well as raising alarms when defects aredetected.

FIG. 2 is an exemplary flowchart 100 showing sample steps which may befollowed in performing automated defect detection using the ABI system2. As shown, after starting at step 102, the process proceeds to step104, in which an initial set of images of members of a device may bereceived by the processing unit 14 from the image capture device 10. Inone embodiment, the device may be an engine 4 and the members may beblades 8 within the engine 4. The blades 8 may be located within a stage6 of the engine 4. The images may be video images. The set of images maybe sequential in terms of the order in which they are captured by theborescope (e.g. image one followed by image two, etc.). In otherembodiments, the images may be non-sequential with regard to the orderin which the images were captured by the image capture device 10. Forexample, every third image captured by the image capture device 10 maybe received by the processing unit 14.

The members may be rotating in the device. For example, the blades 8 mayrotate toward or away from the image capture device 10 when the imagesare being captured. The images captured may be of the same blade 8 indifferent positions in the field of view of the image capture device 10and/or may be of a plurality of blades 8 in different positions in thefield of view of the image capture device 10. Thus, there may beperiodic or semi-periodic motion in the recorded videos of suchinspected engine blades 8.

In step 106 the processing unit may apply Robust Principal ComponentAnalysis (RPCA) to decompose the initial set of images received by theprocessing unit 14 from the image capture device 10 into a first seriesof low rank component images (low rank matrix) and a second series ofsparse component anomaly images (sparse matrix). Using the RPCAtechnique, the initial series of images are decomposed into a low rankmatrix and a sparse matrix utilizing the mathematical equation below.

min_(A,E) ∥A∥ _(x) +λ∥E∥ ₁ s.t. D=A+E

In the equation above, D is the original image data arranged in a matrixof dimension (Height×Width)×Number of Frames. The matrix A is an outputimage sequence of the same size as D. The matrix A has a distinctivecharacteristic of being low rank. The matrix A may be visualized as animage sequence when it is represented as Number of Frames frames of size(Height×Width). This low rank part is determined by minimizing thematrix nuclear norm which is the convex relaxation of rank. E is anotheroutput image sequence of the same size as D and has another distinctivecharacteristic of being sparse. The matrix E may be visualized as animage sequence when it is represented as Number of Frames frames of size(Height×Width). The parameter λ is the weight factor on the sparsecomponent. Each image from image capture device 10 is one column in thelow rank matrix and in the sparse matrix. Other mathematicalformulations of RPCA and algorithms for its solution are known in theart and may be used equivalently in step 106.

Typically blades 8 of an engine 4 are of the same size in a given enginestage 6. When a second blade 8 rotates to the same position as thatwhich the first blade 8 had been in previously, the two images taken atthe two different instances are generally almost the same. Therepetitive, nearly identical images are captured in the A matrix of theequation above. The damaged areas, for example nicks or dents, tend tooccupy a small percentage of the entire image and are captured in thesparse matrix E.

FIG. 3 illustrates an exemplary initial set of video images 24decomposed into a first series 26 of low rank component images 28 and asecond series 30 of sparse component images 32. As can be seen in FIG.3, the defect portion 34 of a blade 8, for instance, is represented bythe sparse component image 32, is generally a relatively small portionof the respective initial image 24.

After separating D, the initial image data matrix, into A, the low-rankpart, and E, the sparse part, additional defect processing may beapplied in step 108 to process the data in the E (sparse) matrix (thesparse component images (32)) in order to further confirm whether thedata in the sparse matrix correspond to physical damage. An example ofsuch additional processing done on the series 30 of sparse componentimages 32 in the E matrix may include statistical techniques such aspolynomial curve fitting, blob extraction and size filtering,morphological filtering, and the like to detect non-smooth edges, tofilter out small regions and sporadic pixels etc. Because only thesparse component image 32 of the initial image 24 content undergoes thisfurther processing, defects can be detected much faster and morereliably using algorithms and methods know in the art.

FIGS. 4A-D illustrate one possible example of additional defectprocessing that may be applied to the sparse component images 32 in thesparse matrix. FIG. 4A illustrates an exemplary initial image 24. FIG.4B illustrates a sparse component image 32 of the initial image 24. Inthe initial image 24, it can be seen that an edge 36, in this example aleading edge, appears to be deformed or non-smooth. In FIG. 4B, it canbe seen that this potential defect is captured in the sparse componentimage 32. As part of the processing of the data represented by sparsecomponent image 32, the curve representing the potentially defectiveedge 36 may be thinned in preparation for curve fitting. FIG. 4Cillustrates an example of the thinned edge 38. The thinned curve or edge38 is then mapped or “fit” to a polynomial curve 40 that approximatesthe normal edge of the blade 8. In this example, the polynomial is asecond order polynomial. If the difference between the polynomial curveand the thinned edge exceeds a predetermined threshold, the blade 8 maybe flagged as defective or reported as defective. FIG. 4D illustratessuch mapping or fitting between the approximation polynomial curve 40and the thinned edge 38 derived from the sparse component image 32. Inother embodiments, other types of curves for fitting may be used as wellcorresponding to the properties of curves in the sparse component.

Yet another example of additional processing on sparse component images32 that may be performed is what is known in the art as blob extractionor size and shape filtering. The shape filtering may be based on theaspect ratio of the blob. FIGS. 5A-C illustrate one example of thisprocessing. FIG. 5A shows an initial image 24 of a blade 8. FIG. 5Billustrates the sparse component image 32 for the initial same bladeimage 32. In the sparse component image 32 a number of potential defectscan be seen. Defects such as dents tend to be larger than otheranomalies in the sparse component image 32 that may be caused by carbondeposits, sporadic pixels, and the like. As such, size and shapefiltering, as is known by those of skill in the art, may be applied tofilter out from the sparse component image 32 those anomalies that fallbelow a predetermined size threshold. FIG. 5C illustrates a filteredsparse component image 32 on which there is one dent remaining aftersize and shape filtering.

FIGS. 4-5 illustrate just a few examples of further defect processingthat may be done on the sparse component image 32. Other types ofadditional processing and filtering, as are known in the art, may beapplied instead of or in addition to such processing on the sparsecomponent images 32 to further automatically analyze potential defects.

After finding defects at step 108, those defects may be reported at astep 110. As discussed above, reporting the defects may involve raisingalarms to alert personnel to further inspect, replace or fix thedefective blade 8. In addition to reporting the defects at the step 110,the defects may also be recorded in the storage medium at a step 112 forfuture reference. The process then ends at a step 114.

INDUSTRIAL APPLICABILITY

In general, the present disclosure sets forth a computer program productand method for performing automated defect detection. The method mayinclude providing storage medium for storing data and programs used inprocessing video images, providing a processing unit for processing suchthe video images of the blades of an engine captured and transmitted bya borescope, and using the processing unit to decompose each of theimages using Robust Principal Component Analysis into a low rank matrixand a sparse component matrix. The method may further include furtherprocessing of image data in the sparse matrix that indicates a possibledefect in order to provide further assurance of the presence of a defectin the plurality of blades. The method may also include applying thedescribed process to other component(s) or mechanical systems.

The present disclosure provides for an automated visual inspection usingautomatic image analysis in which human involvement, required a prioriknowledge, and manual parameter tuning is minimized, thereby minimizinghuman related errors and improving inspection reliability and speed.Also, the present disclosure teaches defect detection using astatistical anomaly detection program and then processing the datafurther that has been identified as a potential defect instead ofprocessing an entire image searching for a wide range of defects.

While only certain embodiments have been set forth, alternatives andmodifications will be apparent from the above description to thoseskilled in the art. These and other alternatives are consideredequivalents and within the spirit and scope of this disclosure and theappended claims.

What is claimed is:
 1. A method of performing automated defectdetection, the method comprising: providing a storage medium for storingdata and programs used in processing images; providing a processing unitfor processing the images; receiving from an image capture device aninitial set of images of a plurality of members inside of a device; andusing the processing unit to apply Robust Principal Component Analysisto decompose the initial set of images into a first series of low rankcomponent images and a second series of sparse component images, whereinthere are at least two images in the initial set.
 2. The method of claim1, wherein the images in said initial set of images are not sequential.3. The method of claim 1, wherein there is more than one image of eachmember in the initial set of images.
 4. The method of claim 1, in whichthe initial set of images includes a first image of a first member and asecond image of a second member, wherein the first image and the secondimage are at different positions in the field of view.
 5. The method ofclaim 1, wherein the plurality of members inside the device arerotating.
 6. The method of claim 1, wherein the first series is in a lowrank matrix and the second series is in a sparse matrix.
 7. The methodof claim 1, wherein the device is an engine and each member is a bladewithin a stage of the engine.
 8. The method of claim 1, furthercomprising determining whether there are defects within the plurality ofmembers using the second series.
 9. The method of claim 1, furthercomprising processing the second series of images to detect edge defectsin one of the members using curve fitting.
 10. The method of claim 1,further comprising using one of size and shape filtering on the secondseries of images to detect whether there is a defect in a member.
 11. Amethod of performing automated defect detection on blades in an aircraftengine, the method comprising: providing a storage medium for storingdata and programs used in processing video images; providing aprocessing unit for processing the video images; receiving from an imagecapture device video images of a plurality of the blades of the engine;using the processing unit to decompose each of the video images usingRobust Principal Component Analysis into a low rank matrix and a sparsematrix; and utilizing video image data in the sparse matrix to determinewhether there are possible defects within the plurality of blades. 12.The method of claim 11, further comprising processing the video imagedata in the sparse matrix to detect whether there is a potential edgedefect in any of the blades.
 13. The method of claim 12, wherein theprocessing step further includes fitting a second order polynomial curveto a potential edge defect.
 14. The method of claim 11, furthercomprising processing the video image data in the sparse matrix usingsize filtering to detect whether there is a defect in any of the blades.15. The method of claim 11, wherein the sparse matrix represents videoimages of potential defects.
 16. The method of claim 11, wherein thevideo images are non-sequential video images.
 17. The method of claim11, wherein there are at least two blades in the plurality of blades.18. A computer program product, comprising a computer usable mediumhaving a computer readable program code embodied therein, the computerreadable program code adapted to be executed to implement a method forperforming automated defect detection on blades in an aircraft engine,the method comprising: receiving from an image capture device videoimages of a plurality of the blades of the engine; and decomposing eachof the video images using Robust Principal Component Analysis into a lowrank matrix and a sparse matrix, wherein data in the sparse matrix isindicative of possible defects in the blades.
 19. The computer programproduct of claim 18, wherein the defects may be one of edge defects anddents.
 20. The computer program product of claim 18, wherein the videoimages are non-sequential video images.